Internet Protocol security (IPSec) is a protocol suite that provides mechanisms for authenticating and encrypting data flowing within a network such as a Virtual Private Network (VPN).
Authentication Header (AH) and Encapsulating Security Payload (ESP) are wire-level protocols provided by IPSec to authenticate (AH) and encrypt (ESP) data. AH may be used in tunnel mode or transport mode.
Transport mode provides a secure connection between two endpoints as it encapsulates the payload portion of Internet Protocol (IP) packets sent over the secure connection. With tunnel mode, the entire IP packet is encapsulated thereby to provide a virtual secure hop between the two endpoints.
FIG. 1A shows the contents of an IP datagram 100 that has undergone IPSec AH authentication to include an AH header. FIG. 1B shows the components of the AH header 102 in greater detail. It will be seen that the AH Header 102 includes authentication data 104. The data 104 is usually a cryptographic hash-based message authentication code computed over nearly all fields of the original IP packet save for those that are modified in transit. The fields modified in transit include TTL and header checksum. FIG. 1C shows the IP datagram 100 with the fields that are protected by AH Authentication shaded. The data 104 carries an integrity check value (ICV) which may be a MD5 or SHA-1 hash.
Network Address Translation (NAT) is a technology that is used to map a range of private addresses to and from a (usually) smaller set of public addresses. This reduces the demand for routable public IP space. NAT devices work by modifying IP headers associated with an IP datagram on the fly. In particular the source and/or destination IP addresses are changed. When a source or header IP address is changed, it forces a recalculation of the header checksum. This has to be done anyway, because the NAT device typically serves as one “hop” in the path from source to destination, and this requires the decrement of the TTL (Time To Live) field. However, as noted above since the TTL and header checksum fields are always modified in flight, AH knows to excludes them from coverage, but this does not apply to the IP addresses. These are included in the integrity check value, and any modification will cause the check to fail when verified by the recipient. FIG. 1D shows the IP datagram 100 with the fields that are modified by NAT shaded and FIG. 1E shows the IP datagram 100 with the fields that are broken by NAT shaded.
Because the ICV incorporates a secret key which is unknown by intermediate parties, the NAT router is not able to recalculate the ICV, making NAT incompatible with IPSec AH authentication.